{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 自主避障 - 演示实例\n",
    "\n",
    "在此例子中，我们将使用训练有素的模型来检测机器人是“自由zhuan”还是“受阻”，以实现机器人上的防撞行为。 \n",
    "\n",
    "## 加载模型\n",
    "\n",
    "我们假设您已经按照训练模型示例中的说明将``best_model.pth''下载到了当前目录下。 现在，您应该将此模型上传到此笔记本的\n",
    "使用Jupyter Lab上传工具创建目录。 完成后，在此笔记本的目录中应该有一个名为“ best_model.pth”的文件。\n",
    "\n",
    ">在调用下一个单元格之前，请确保文件已完全上传      \n",
    "> 若您并未训练模型，此处我们提供已训练好的模型下载链接                \n",
    ">    百度云：https://pan.baidu.com/s/1ARb7ZN-bGxY1fj1pcyRJGQ 提取码：ft7q\n",
    "\n",
    "执行以下代码以初始化PyTorch模型。 从训练模型示例中看起来应该非常熟悉。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "\n",
    "model = torchvision.models.alexnet(pretrained=False)\n",
    "model.classifier[6] = torch.nn.Linear(model.classifier[6].in_features, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，从您上传的``best_model.pth''文件中加载训练后的权重"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.load_state_dict(torch.load('best_model.pth'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当前，模型权重位于CPU内存上，执行以下代码以传输到GPU设备。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device('cuda')\n",
    "model = model.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建预处理功能\n",
    "\n",
    "现在，我们已经加载了模型，但是有一个小问题。 我们训练模型的格式与摄像机的格式“不完全”匹配。 要做到这一点，\n",
    "我们需要做一些*预处理*。 这涉及以下步骤\n",
    "\n",
    "> 1.从BGR转换为RGB                                                                                 \n",
    "> 2.从HWC布局转换为CHW布局                                                                       \n",
    "> 3.使用与训练期间相同的参数进行归一化（我们的相机提供[0，255]范围内的值，并训练[0，1]范围内的加载图像，因此我们需要按255.0缩放                                                                 \n",
    "\n",
    "> 4.将数据从CPU内存传输到GPU内存                                                                  \n",
    "> 5.添加批次尺寸           "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很好！ 现在，我们定义了预处理功能，该功能可以将图像从相机格式转换为神经网络输入格式。\n",
    "\n",
    "现在，让我们开始展示相机。 您现在应该已经对此非常熟悉。 我们还将创建一个滑块，该滑块将显示\n",
    "机器人被阻止的可能性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import traitlets\n",
    "from IPython.display import display\n",
    "import ipywidgets.widgets as widgets\n",
    "from jetbot import Camera, bgr8_to_jpeg\n",
    "\n",
    "camera = Camera.instance(width=224, height=224)\n",
    "image = widgets.Image(format='jpeg', width=224, height=224)\n",
    "blocked_slider = widgets.FloatSlider(description='blocked', min=0.0, max=1.0, orientation='vertical')\n",
    "\n",
    "camera_link = traitlets.dlink((camera, 'value'), (image, 'value'), transform=bgr8_to_jpeg)\n",
    "\n",
    "display(widgets.HBox([image, blocked_slider]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们还将创建机器人实例，以驱动电机。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from jetbot import Robot\n",
    "\n",
    "robot = Robot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们将创建一个函数，只要相机的值发生变化，该函数就会被调用。 此功能将执行以下步骤\n",
    "\n",
    "1.预处理相机图像   \n",
    "2.执行神经网络      \n",
    "3.当神经网络输出表明我们被阻止时，我们将向左转，否则我们将前进。   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import torch.nn.functional as F\n",
    "import time\n",
    "\n",
    "def update(change):\n",
    "    global blocked_slider, robot\n",
    "    x = change['new'] \n",
    "    x = preprocess(x)\n",
    "    y = model(x)\n",
    "    \n",
    "    # we apply the `softmax` function to normalize the output vector so it sums to 1 (which makes it a probability distribution)\n",
    "    y = F.softmax(y, dim=1)\n",
    "    \n",
    "    prob_blocked = float(y.flatten()[0])\n",
    "    \n",
    "    blocked_slider.value = prob_blocked\n",
    "    \n",
    "    if prob_blocked < 0.5:\n",
    "        robot.forward(0.4)\n",
    "    else:\n",
    "        robot.left(0.4)\n",
    "    \n",
    "    time.sleep(0.001)\n",
    "        \n",
    "update({'new': camera.value})  # we call the function once to intialize"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很好！ 我们已经创建了神经网络执行功能，但是现在我们需要将其附加到相机上进行处理。\n",
    "\n",
    "我们通过``observe''函数来实现这一目标。\n",
    "\n",
    ">警告：此代码将移动机器人！ 请确保您的机器人有间隙。 避免碰撞应该起作用，但是神经\n",
    ">网络仅取决于其训练的数据！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.observe(update, names='value')  # this attaches the 'update' function to the 'value' traitlet of our camera"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "太棒了！ 如果您的机器人已插入电源，则现在应该在每个新的摄像机框架上生成新命令。 也许首先将机器人放在地面上，看看它遇到障碍时会做什么。\n",
    "\n",
    "如果要停止此行为，可以通过执行以下代码来取消附加此回调。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "camera.unobserve(update, names='value')\n",
    "\n",
    "time.sleep(0.1)  # add a small sleep to make sure frames have finished processing\n",
    "\n",
    "robot.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也许您希望机器人在不将视频流传输到浏览器的情况下运行。 您可以如下所示断开相机的链接。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera_link.unlink()  # don't stream to browser (will still run camera)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要继续流式传输，运行以下内容."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera_link.link()  # stream to browser (wont run camera)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结论\n",
    "\n",
    "就是这个现场演示！ 希望您玩得开心，机器人可以聪明地避免碰撞！\n",
    "\n",
    "如果您的机器人不能很好地避免碰撞，请尝试找出失败的地方。 这样做的好处是，我们可以针对这些故障场景收集更多数据\n",
    "并且机器人应该变得更好：)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
